gh-98894: Fix dtrace tests in shared builds - #153372
Open
stratakis wants to merge 2 commits into
Open
Conversation
Contributor
Author
|
cc @vstinner |
Generate SystemTap probe definitions targeting libpython for shared builds and use centralized USDT probe object discovery for readelf and BPFTrace.
Contributor
Author
|
Rebased to fix conflicts |
Member
|
Python configured with: Without the change, I get: |
Contributor
Author
|
The systemtap tests require the same kernel-devel package as the running kernel, if it's not there the test will be skipped. |
Member
|
I tested again the change. Python built with Without the change: With the change: So we get: "Total tests: run=3 skipped=6" => "Total tests: run=9 skipped=4". 6 more tests are run thanks to the change. With the change, BPFTraceNormalTests and BPFTraceOptimizedTests tests are run. |
vstinner
reviewed
Aug 6, 2026
| PROBE_PLACEHOLDER = "@PYTHON_SYSTEMTAP_PROBE@" | ||
|
|
||
| @staticmethod | ||
| def _quote_systemtap_string(value): |
Member
There was a problem hiding this comment.
There is no need to make the 3 added methods as private: you can remove the "_" prefix.
Comment on lines
+231
to
+236
| executable = self._quote_systemtap_string(sys.executable) | ||
| probe_binary = get_probe_binary() | ||
| if probe_binary != sys.executable: | ||
| probe_binary = self._quote_systemtap_string(probe_binary) | ||
| return f'process("{executable}").library("{probe_binary}").mark' | ||
| return f'process("{executable}").mark' |
Member
There was a problem hiding this comment.
Suggested change
| executable = self._quote_systemtap_string(sys.executable) | |
| probe_binary = get_probe_binary() | |
| if probe_binary != sys.executable: | |
| probe_binary = self._quote_systemtap_string(probe_binary) | |
| return f'process("{executable}").library("{probe_binary}").mark' | |
| return f'process("{executable}").mark' | |
| executable = self._quote_systemtap_string(sys.executable) | |
| probe_binary = get_probe_binary() | |
| if probe_binary == sys.executable: | |
| return f'process("{executable}").mark' | |
| # Python built with --enable-shared | |
| probe_binary = self._quote_systemtap_string(probe_binary) | |
| return f'process("{executable}").library("{probe_binary}").mark' |
Comment on lines
+238
to
+242
| def _render_script(self, script_file): | ||
| with open(script_file) as script: | ||
| return script.read().replace( | ||
| self.PROBE_PLACEHOLDER, self._python_probe() | ||
| ) |
Member
There was a problem hiding this comment.
Suggested change
| def _render_script(self, script_file): | |
| with open(script_file) as script: | |
| return script.read().replace( | |
| self.PROBE_PLACEHOLDER, self._python_probe() | |
| ) | |
| def _render_script(self, filename): | |
| with open(filename) as fp: | |
| script = fp.read() | |
| return script.replace(self.PROBE_PLACEHOLDER, self._python_probe()) |
| check_returncode=check_returncode, | ||
| ) | ||
| finally: | ||
| os.unlink(generated_script_file) |
Member
There was a problem hiding this comment.
Please add from test.support import os_helper at the top, and replace this line with:
Suggested change
| os.unlink(generated_script_file) | |
| os_helper.unlink(generated_script_file) |
os_helper.unlink() tries harder to remove a file :-)
| return binary | ||
|
|
||
|
|
||
| def truncate_output(output, *, max_lines=3, max_chars=500): |
Member
There was a problem hiding this comment.
This PR only changes many functions. I would prefer to revert this truncate_output() change (remove the function, restore old code).
I'm not convinced that truncating the output is a good thing. On our CI, it's usually hard to reproduce locally a failure, and so we need to collect as much data as we want when an error occurs. I would prefer not truncating the output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When Python is configured with --with-dtrace and --enable-shared, its USDT probes are in libpython, not the executable. The SystemTap and BPFTrace backends in test_dtrace were searching the executable, causing the tests to fail or time out. This PR makes both backends target the object containing the probes. Static builds are unaffected.
On Fedora, this can be reproduced by running test_dtrace for BPFTrace as sudo and for SystemTap either as sudo or as a user in the stapusr and stapdev groups.
Also added a commit to truncate the output of the tools as it can be quite verbose when it errors out (you can inspect by running the same tests without sudo).